Method of Servicing Requests to Manage Network Congestion and Server Load and Server Thereof

ABSTRACT

A method of servicing requests for information at a server includes receiving a request for information from a client device, and determining a number of attempts of the request made by the client device. Both the client device and the server may count the number of attempts for each request. When the number of attempts is greater than a threshold, the method includes servicing the request and sending the information to the client device. When the number of attempts is not greater than the threshold, the method includes sending a service unavailable message to the client device. A retry-after delay may be calculated by the server according to a predicted time for the server to exit the overloaded state and included in the service unavailable message. The predicted time may take into account both requests currently being serviced and requests that have been scheduled to be serviced in the future.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention pertains generally to computer servers. More specifically,the invention relates to managing network congestion and server load ina system having a plurality of client devices such as a hospitalitymedia system.

2. Description of the Prior Art

In a typical hotel media system, set-top boxes (STBs) are installed inguest rooms and are in communication with a central server. Thecommunication is usually performed over an Internet protocol (IP)network available in the hotel, and part of a boot-up process at eachSTB typically involves requesting configuration information from theserver. A problem with such a configuration is that the central servermay become overloaded with simultaneous requests from many STBs. Oneexample of a particularly problematic event is when power is restored inthe hotel after an unexpected outage. In this situation, STBs in eachroom will boot-up at approximately the same time, and, when the hotelincludes hundreds or even thousands of STBs, the resulting sudden surgein network requests and server load can be overwhelming. The entiresystem may be exponentially delayed as a result.

A common solution to this problem is to introduce a random delay at eachSTB before attempting to request information from the server. Randomdelays tend to spread out the requests from STBs and allow the servermore time to process each request. In the event that the server stillbecomes overloaded and unable to service all the requests, affected STBsmay wait another random delay as a back-off delay before attempting therequest again. The back-off delays may exponentially increase induration to further spread out the requests and allow the server moretime to recover.

Although this solution works well in a system with a relatively smallnumber of STBs, it does not scale to hotels having thousands of STBs.One reason is that the random numbers generated by the STBs may notactually be random and therefore a large number of STBs may “randomly”chose exactly the same delay. Even with true random delays, there isstill a chance that hundreds of STBs will all choose the same (orsimilar) random numbers. When this happens, the server unfortunatelyremains overloaded and will need to reject some requests. These rejectedrequests will be further spread out by increasing the upper-bound on theback-off time delay at each STB. In a large hotel, the upper-bound ofthe random delay needs to quickly grow in order to sufficiently spaceout the requests in the event that all STBs are requesting data at thesame time. Minutes of back-off delay may be encountered in the eventthat thousands of STBs are all rebooting at the same time. However, inthe event that there is actually no long-term server load, it is veryundesirable to have exponentially increasing delays. For example, whenonly a single STB is rebooted but then happens to be unlucky on itsfirst few attempts at requesting configuration data from the server, theSTB may randomly choose a very long back-off delay. In the event thatthe server was only momentarily busy, the long back-off delay at thesingle STB is unnecessary and a waste of user's time.

SUMMARY OF THE INVENTION

According to one aspect of the invention there is provided a method ofservicing requests for information at a server. The method is performedby the server and includes receiving a request for information from aclient device; determining a number of attempts of the request made bythe client device; servicing the request and sending the information tothe client device when the number of attempts is greater than athreshold; and sending a service unavailable message to the clientdevice when the number of attempts is not greater than the threshold.

According to another aspect of the invention, there is provided a serverfor servicing requests for information. The server includes acommunication module being coupled to a network and receiving a requestfor information from a client device via the network, and a requestnumber module for determining a number of attempts of the request madeby the client device. A servicing module services the request and sendsthe information to the client device when the number of attempts isgreater than a threshold, and sends a service unavailable message to theclient device when the number of attempts is not greater than thethreshold.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram of a hotel media system including a mediasystem server and a plurality of client devices according to oneconfiguration of the present invention.

FIG. 2 illustrates a table showing actions taken by the server of FIG. 1in different situations according to an exemplary configuration of thepresent invention.

FIG. 3 illustrates a simplified hotel media system having three STBsthat have all been rebooted at substantially the same time.

FIG. 4 illustrates a first exemplary timeline diagram for the simplifiedmedia system of FIG. 3 where STBs make requests at substantially thesame time.

FIG. 5 illustrates a second exemplary timeline diagram for thesimplified media system of FIG. 3 where STBs make requests atsubstantially the same time.

FIG. 6 is a flowchart describing operations performed by the server ofFIG. 1 when servicing requests for information according to apredetermined threshold representing the maximum number of acceptableattempts per request.

FIG. 7 is a flowchart describing operations performed by the server ofFIG. 1 when servicing requests for information according to a dynamicthreshold representing the maximum number of acceptable attempts perrequest.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of a hotel media system 100 including a mediasystem server 102 and a plurality of client devices 104 according to oneconfiguration of the present invention. For illustration purposes, theclient devices 104 are shown as set-top boxes (STBs); however, thepresent invention is equally applicable to other types of clientdevices. The STBs 104 are coupled to the server 102 through a network106, which could be a wired or wireless Internet protocol (IP) networkinstalled at the hotel property or could be an external network such asthe Internet. In one configuration, the server 102 includes acommunication module 112, a servicing module 116, a request numbermodule 118, a threshold setting module 120, a delay time predictionmodule 122, a memory device 124, a configuration database 130, and aprocessor 114. Although, the modules 112, 116, 118, 120, 122 are shownin FIG. 1 as a dedicated hardware modules, they may also be implementedin other configurations as software program(s) that are executed by ageneral or specific purpose processor 114 and that causes the processor114 to operate pursuant to the software program(s) to perform thefunctions described below.

At boot-up, each STB 104 executes a stored application program 110 thatcauses a request for configuration information from the server 102. Inanother configuration, the requested configuration information may bethe application program 110 (or a significant portion of the applicationprogram 110) itself. This could be the case where that STBs 104 run abuilt-in web browser upon boot-up and then download, from the server102, a custom application program 110 that is thereafter executed withinthe web browser environment on the STB 104. The server 102 services eachrequest for configuration information by searching for the requestedinformation in the configuration database 130 and then sending therequested information to the requesting STB 104. The informationrequests by the STBs may be dynamic in nature such that differentinformation is sent from the server 102 to different STBs 104.Retrieving the desired information for each STB 104 in the database 130may incur significant delays while the database 130 searches for andthen formats the requested information. Caching data retrieved from thedatabase 130 may somewhat speed the servicing times; however, there willinevitably be a load on the server 102 for a period of time due to eachrequest.

Depending on certain factors, the server 102 may determine itself to bein an overloaded state. One factor that may be utilized to determinewhether or not the server 102 is overloaded includes monitoring how manysimultaneous connections at the server 102 are currently open with STBs104. For example, a typical transport control protocol (TCP)/IP basedmedia system server 102 may only be able to support a predeterminedmaximum number of open TCP connection (e.g., one thousand open TCPconnections) and will be unable to open any additional networkconnections once the limit is reached. Therefore, the server 102 maymonitor how many connections are currently open and define its operatingstate to be overloaded when the number of currently open connections isgreater than a percentage such as ninety percent of the maximumallowable connections, for example. Another factor that could beutilized to determine whether or not the server 102 is overloaded,either in combination with open connections or separately, includesmonitoring an average delay-time per request. The server 102 maycontinuously calculate an average time delay between receiving requestsfrom STBs 104 and providing the requested information to the STBs 104.When the average time becomes greater than a threshold 117 such as tenseconds, for example, the server 102 may define its operating state tohave become overloaded. Other combinations of factors may also beutilized to determine when the server 102 has become overloadedincluding, but not limited to, processor 114 usage, communication module112 network usage, database 130 usage, etc.

When operating in an overloaded state, in one configuration, unless therequesting STB 104 has already reached a maximum number of acceptablerequest attempts, the server 102 refuses to service newly receivedrequests from the STB 104. By rejecting newly received requests, verylittle additional load is placed on the server 102. For example, theserver 102 may refuse the connection with a service unavailable responsecode such as hypertext transfer protocol (HTTP) response code 503Service Unavailable as defined by RFC 2616, page 70. However, bycontinuing to service requests that have already been attempted morethan the maximum number of acceptable times, a minimum service qualityis preserved. For example, in one configuration, when receiving arequest from a STB 104, the request number module 118 determines anumber of attempts of the request made by the STB 104, and, whenoverloaded, the servicing module 116 only services the request when thenumber of attempts is greater than a threshold 117 that represents amaximum number of acceptable attempts. As will be explained, thethreshold 117 may be predetermined or dynamic.

To allow the request number module 118 to determine the number ofattempts, each STB 104 may include a request attempt counter 108 forcounting request attempts. When making a request, the current number ofattempts is included in the request. For example, upon a first attemptof a request, the counter would indicate a value of “1”. If the requestwas retried, for example due to the first attempt being refused or notreceived by the server 102, the counter would be incremented and wouldindicate a value of “2” since this is the second attempt. The samecounter action would continue for each subsequent request until therequest is finally serviced by the server 102 and the counter 108 wouldthen be reset back to a value of “1” for the first attempt of a newrequest. For an HTTP based request, the STB 104 may include the numberof attempts within the request in a custom header such as “X-attempts”.The request number module 118 then determines the number of attempts ofthe request by examining the headers of the request to read the currentnumber of attempts as specified within the “X-attempts” header. Fornon-HTTP based requests, a similar field may be created in a header ordata section of the request packet. In this way, the STBs 104 may keeptrack of the number of attempts for each request, which reduces load onthe server 102 by not requiring the server 102 to track the number ofattempts per request. This configuration is particularly well-suited toa hotel media system 100 having STBs 104 being provided as a part of thesystem 100. Because the STBs 104 are only executing authorizedapplication program(s) 110 or firmware that is designed to cooperatewith the server 102, there is little concern that a particular STB 104will “cheat” by inaccurately reporting a higher than accurate number ofrequest attempts in order to receive preferential service from theserver 102.

In another configuration, the memory device 124 of the server 102 storesrequest attempt counters 126 for different STBs 104. The request numbermodule 118 then determines the number of attempts of each request byutilizing the counter 126 corresponding to the requesting STB 104 tocount a number of request attempts made by the requesting set-top box104. Although, this configuration places an additional load on theserver 102 to keep track of the number of attempts for each STB 104, onebenefit is that fairness is guaranteed because it will be impossible fora rogue STB 104 to report an inaccurate number of attempts to gainpreferential treatment from the server 102. This is useful when thepresent invention is utilized with client devices that cannot be trustedto accurately report the correct number of attempts, or when clientdevices are provided by or under the control of users or a third-partyvendor. For example, in a media system 100 that allows foreign devicesto request information from the server 102, the foreign devices may notbe configured to include the number of attempts within the header ofeach request. In this case, the server 102 may track the number ofattempts using the counters 126 in the memory device 124.

In another configuration, the server 102 may automatically detect whichclient devices are including the number of attempts in the requestheaders, and only track the number of attempts for client devices thatare not already operable to do it themselves. This allows the server 102to utilize the present invention with any kind of client device whetheror not the client device natively supports the method of the presentinvention.

FIG. 2 illustrates a table showing actions 200, 202, 204, 206 taken bythe server 102 of FIG. 1 in different situations according to anexemplary configuration of the present invention. As shown in FIG. 2,when the server 102 is not operating in the overloaded state (actions200, 204), regardless of the number of attempts for each request, theserver 102 simply accepts the network connection with the requesting STB104 and services the request. Although, there may be slight delays dueto servicing times and multitasking overhead, because the server 102 isnot overloaded, the server 102 generally starts servicing the requestimmediately.

When the server 102 is overloaded (actions 202, 206), whether or not anincoming request will be serviced is decided by the servicing module 116according to how many attempts of the request have been made so far.When the number of attempts is greater than a threshold 117 (action206), this means the requesting STB 104 has already repeated thisrequest more than a maximum acceptable number of times and therefore isgiven preference by the server 102. For example, the requesting STB 104may have been delayed due to being repeatedly refused service by theserver 102. In order to prevent the same STB 104 from being continuouslyrefused service, once the number of attempts has exceeded the threshold117, even if overloaded, the server 102 will accept the networkconnection with the requesting STB 104 and hold the connection openuntil the server 102 is able to service the request and send therequested information to the STB 104. Because the server 102 isoverloaded, there may still be a delay while the server 102 services therequest, but a delay with guaranteed service at the end of the delay isbetter than continuing to refuse service to a STB 104 that has alreadyattempted the request greater than the threshold 117 number of times.Additionally, to speed service, the servicing module 116 may internallychange the job order to give priority to a request that has a number ofattempts being greater than the threshold 117.

As shown by action 202 in FIG. 2, when the server 102 is overloaded andreceives a request having a number of attempts being less than or equalto the threshold 117, the server 102 refuses the connection by sending aservice unavailable message. Additionally, in order to prevent aworsening of the overloaded state and to avoid unnecessary networkcongestion from request retries, the delay time prediction module 122predicts an amount of time required by the server 102 to exit theoverloaded state, and includes this time as a retry-after delay in theservice unavailable message. Upon receiving the service unavailablemessage from the server 102, the requesting STB 104 waits theretry-after delay specified in the service unavailable message beforeattempting a subsequent retry of the request. When using the HTTPresponse code 503 Service Unavailable, as specified by RFC 2616, page70, a retry-after delay field is available for specifying theretry-after delay. However, it should be noted that no guidance isprovided by RFC 2616 on how to calculate the retry-after delay. When notusing HTTP requests, a custom service unavailable packet may be definedincluding a similar retry-after delay field.

To calculate the retry-after delay, in one configuration of the presentinvention, the delay time prediction module 122 includes a pendingrequest log 132 storing a record of pending requests including currentrequests being serviced and prior requests from other STBs 104 that havenot yet been serviced and for which the servicing module 116 previouslysent one or more service unavailable messages. The delay time predictionunit 122 predicts the amount of time required to exit the overloadedstate by calculating an estimated time required to service these pendingrequests. In this way, when refusing service, the retry-after delay isused by the server 102 to schedule the subsequent retry to occur whenthe server 102 will exit the overloaded state. Request retries arethereby scheduled to maximize utilization of the server 102 withoutfurther overloading the server 102. By refusing the connections forrequests for which the number of attempts is not greater than thethreshold 117, and scheduling them to be retried when the server 102predicts it will no longer be overloaded, the server 102 will not reachits maximum number of possible connections, and will therefore continueto be available for more urgent requests that have a number of attemptsalready exceeding the threshold 117 (action 206 in FIG. 2). Whencalculating the retry-after delay, a random time may also be added to orsubtracted from the predicted time to exit the overloaded state toaccount for unknown server 102 events that may occur before thesubsequent retry of the request is made.

The threshold 117 may be predetermined or dynamically configured by thethreshold setting module 120. In one configuration, the threshold 117represents a maximum allowable number of attempts per request before theserver 102 will always accept the connection and service the request,even when it is overloaded. The threshold 117 may be configured to limitthe maximum delay that could be encountered by a very unlucky STB 104before the server 102 will accept the connection and begin processingthe request. For example, if N designates the total number of STBs 104,P designates the number of simultaneous requests that may be serviced bythe server 102, S designates a maximum servicing time per request, and Tdesignates the threshold 117 (i.e., the maximum acceptable number ofrequest attempts, after which the server 102 will accept the connectioneven if it is overloaded), the maximum delay (D) before the server 102will accept a connection request from a STB 104 can be calculated by theformula:

$\begin{matrix}{D = {\left( {N - 1} \right)\frac{TS}{P}}} & {{Formula}\mspace{14mu} 1}\end{matrix}$

By adjusting the value of the threshold 117 (T), an acceptable maximumdelay (D) may be configured. For example, in a system 100 includingthree thousand STBs 104, where the server 102 can simultaneously handleone thousand TCP connections, and each request could take up to 500 msto service, a threshold 117 (T) value of “10” attempts would limit themaximum delay (D) to approximately 15 seconds before the server 102would, even if currently overloaded, accept an eleventh connectionattempt from a very unlucky STB 104 and begin servicing the request. Thethreshold 117 may also be dynamically set by the server 102 whenparameters in the above formula change. For example, to maintain anacceptable delay (D) when parameters such as the number of active STBs104 (i.e., in use by guests) or the number of simultaneous requests thatmay be serviced by the server 102 change (i.e., due to scheduledmaintenance processes using some of the connections), the threshold 117may be automatically adjusted by the threshold setting module 120.

To better illustrate aspects of the present invention, simplifiedexamples of operations of a media system 300 having the server 102 areshown in FIG. 3, FIG. 4, and FIG. 5. The examples shown are simplifiedin that the media system 300 only includes three clients (i.e, STBs 302,304, 306); and the server 102 is assumed to only be able to service asingle request at a time and is determined to be overloaded when it isservicing a request. Each request is also assumed to take the server 102three seconds of time to service. Additionally, the threshold 117utilized by the servicing module 117 is dynamically set to be a value of“1” when the server 102 is overloaded (i.e., currently servicing arequest) and to be set to a value of “0” when the server 102 is idle(i.e., not servicing a request). It should be noted that theseconstraints are artificially low in order to facilitate an easierunderstanding of certain aspects of the present invention. In actualpractice, however, the server 102 may be utilized in conjunction withany number and types of client devices, may be capable of simultaneouslyservicing hundreds or thousands of requests before being determined tohave entered an overloaded state, may have a threshold 117 set to avalue higher than “1”, and may take a different amount of time toservice each request according to various factors such type of request,processor 114 load, database 130 load, etc.

FIG. 3 illustrates a simplified hotel media system 300 having three STBs302, 304, 306 that have all been rebooted at substantially the sametime. In this example, requests for configuration information arereceived at the server 102 at substantially the same time. Such asituation could occur, for example, after an unexpected power outage atthe hotel. Because, in this example, the server 102 is deemed to beoverloaded upon beginning to service the first request 320, the server102 refuses the connection with the second STB 304 by sending a serviceunavailable message having a retry-after delay of three seconds. In thisway, the second request 322 is scheduled to be retried when the server102 has finished processing the first request 320. Likewise, the server102 refuses the connection with the third STB 306 by sending a serviceunavailable message having a retry-after delay of six seconds. In thisway, the third request 324 is scheduled to be retried by the third STB306 when the server 102 has finished processing both the first andsecond requests 320, 322. The pending requests log 132 may be utilizedby the delay time prediction module 122 to keep track of how manypending requests exist and how long each request is estimated to requirefor completion in order to predict the retry-after delay required toallow the server 102 to exit the overloaded state.

Because the server 102 has scheduled the subsequent retries to occurwhen the server 102 predicts it will no longer be overloaded, theservicing time is known to be three seconds, and because the threshold117 of maximum request attempts is “1” in this example, the STBs 302,304, 306 are able to accurately report a message such as “Your mediaexperience is scheduled to begin in XX seconds” to a user. This enhancesuser satisfaction because the delay at each STB 302, 304, 306 is bothknown and minimized given actual server 102 usage. In otherconfigurations, a message indicating the upper-bound of the maximumdelay (D) determined by formula I could be displayed such as “Your mediaexperienced is scheduled to begin in at most XX seconds.” In both cases,the on-screen time delay may count down in real time.

FIG. 4 illustrates a first exemplary timeline diagram for the simplifiedmedia system 300 having STBs that make requests at substantially thesame time. As shown in FIG. 4, at time point 0, there are no requestscurrently being serviced; therefore, the server 102 is not in theoverloaded state, and the threshold 117 is dynamically set to a value of“0”. Additionally, since there are no current requests being serviced orprior requests from other set-top boxes that have not yet been serviced,the pending requests are “0” and the predicted time delay to exit theoverloaded state is also “0”.

At time point 1, two incoming requests are received at substantially thesame time. Both requests have a number of attempts being equal to “1”since they are first attempts. The server 102 accepts the connectionwith the first STB 302 and begins to process the request from the firstSTB 302. Due to the above defined constraints in this example, theserver 102 now determines itself to be in the overloaded state andtherefore refuses the connection and sends a service unavailable messageto the second STB 304. Included in the service unavailable message is aretry-after delay of “3” as determined according to the predicted amountof time the server 102 will take in order to finish servicing therequest from the first STB 302. At the end of time point 1, the pendingrequests value is “2” since both the first and second STBs 302, 304 havemade requests that have not yet been serviced. Additionally, thepredicted time delay is “5” since there are still two time units (i.e.,seconds in this example) left to service the request from the first STB302 and three time units for the scheduled request for STB 304.

At time point 2, a request having a number of attempts of “1” isreceived from the third STB 306. Since the server 102 is servicing therequest from the first STB 302, the server 102 is still deemed as beingoverloaded and therefore refuses the connection because the number ofattempts of the incoming request (“1”) is not greater than the threshold117 of “1”. The server 102 sends a service unavailable message to thethird STB 306 and includes in the service unavailable message aretry-after delay of “5” as determined according to the predicted amountof time the server 102 will take in order to finish servicing both therequest from the first STB 302 and the prior request from the second STB304 that has not yet been serviced and that was scheduled by theservicing module 116 to be retried after the request from the first STB302 is finished being serviced. At the end of time point 2, the pendingrequests value is “3” since all three STBs 302, 304, 306 have maderequests that have not yet been serviced. Additionally, the predictedtime delay is “7” since there is still one time unit remaining toservice the current request from the first STB 302 and then the twoscheduled requests from STB 304, 306 will each take three time units tobe serviced.

At the end of time point 3, the server 102 has finished servicing therequest from the first STB 302 and sends the requested information tothe first STB 302. The pending requests is decreased to a value of “2”since there are now only two prior requests that have not been serviced,and the predicted time delay is decreased to a value of “6” since onetime unit has passed and no new requests were scheduled. In oneconfiguration, the server 102 may now be deemed to no longer beoperating in the overloaded state until a next request is received;however, in this example the overloaded state is shown maintainedbecause a prior request has been scheduled to be received at the verynext time point.

At time point 4, two more incoming requests are received. Because therequest from the second STB 304 has a number of attempts being equal to“2” (i.e., greater than the threshold 117 of “1”), the server 102 givesthis request priority, accepts the connection with the second STB 304,and begins servicing this request. Because the server 102 is operatingin the overloaded state and the request from the first STB 302 does nothave a number of retries being greater than the threshold 117 of “1”,the server 102 refuses the connection with the first STB 302 by sendinga service unavailable message. Included in the service unavailablemessage is a retry-after delay of “6” as determined according to thepredicted amount of time the server 102 will take in order to finishservicing both the currently being serviced request from the second STB304 and the prior request from the third STB 306 that has not yet beenserviced. At the end of time point 4, the pending requests value isagain “3” since all three STBs 302, 304, 306 have made requests thathave not yet been serviced. Additionally, the predicted time delay is“8” since there are still two time units remaining to service thecurrent request from the second STB 304 and then the two scheduledrequests from STB 306, 302 will each take three time units to beserviced.

At the end of time point 5, the predicted time delay is decreased to avalue of “7” since one time unit has passed and no new requests werescheduled.

At the end of time point 6, the server 102 has finished servicing therequest from the second STB 304 and sends the requested information tothe second STB 304. The pending requests is decreased to a value of “2”since there are now only two prior requests that have not been serviced,and the predicted time delay is decreased to a value of “6” since onetime unit has passed and no new requests were scheduled.

At time point 7, the second attempt of the request by the third STB 306is received and the server 102 accepts the connection and beginsservicing the request. The predicted time delay is decreased to a valueof “5” since one time unit has passed and no new requests werescheduled. Note that the second request from the third STB 306 does notcount as a newly scheduled request since this request was alreadypending and the second attempt is now arriving at a time point asscheduled.

At the end of time point 8, the predicted time delay is decreased to avalue of “4” since one time unit has passed and no new requests werescheduled.

At the end of time point 9, the server 102 has finished servicing therequest from the third STB 306 and sends the requested information tothe third STB 306. The pending requests is decreased to a value of “1”since there is now only one prior request that has not been serviced,and the predicted time delay is decreased to a value of “3” because onlyone request remains to be serviced.

At time point 10, the second attempt of the request by the first STB 306is received and the server 102 accepts the connection and beginsservicing the request. The predicted time delay is decreased to a valueof “2” since only two time units are left to be serviced in the currentrequest and no other requests are scheduled.

At the end of time point 11, the predicted time delay is decreased to avalue of “1” since only one time unit is required to service the currentrequest and no other request are scheduled.

At the end of time point 12, the server 102 has finished servicing therequest from the first STB 302 and sends the requested information tothe first STB 302. The pending requests is decreased to a value of “0”since there are no other requests pending, and the predicted time delayis also set to “0” since the server 102 is entering the idle state.

At time point 13, similar to time point 0, the server 102 is idle andready to begin immediately servicing any incoming request.

FIG. 5 illustrates a second exemplary timeline diagram for thesimplified media system 300 having STBs that make requests atsubstantially the same time. FIG. 5 is very similar to FIG. 4, except inFIG. 5, the request made by the third STB 306 at time point 2 has anumber of attempts of “2”, which is higher than the exemplary threshold117 value of “1”. The number of attempts of “2” could have been causedfor any number of reasons including that the server 102 was unable toservice the first attempt (not shown in FIG. 5) because it did notreceive the first attempt due to a network 106 problem. Because therequest has a number of attempts greater than the threshold 117, unlikethat shown in FIG. 4, in FIG. 5 the server 102 does not refuse theconnection to the third STB 306 at time point 2. Instead the server 102holds the connection in an open state while it finishes processing thecurrent request. At time point 4, the server 102 then begins processingthe request made by the third STB 306. This action slightly delays theservicing of the request made by the second STB 304, but because thenumber of attempts for the request by the second STB 304 is also greaterthan the threshold 117, the server 102 holds the connection with thesecond STB 304 open. At the end of time point 6, the server 102 sendsthe requested information to the third STB 306, and at the end of timepoint 9, the server 102 sends the requested information to the secondSTB 304. In this configuration, when a high priority request (defined ashaving a number of attempts greater than the threshold 117) arriveswhile the server 102 is already in an overloaded state, the connectionto the STBs making the high priority request is accepted and held openwhile the server 102 services accepted requests in afirst-come-first-serve basis. In another configuration, the server 102may suspend (or cancel) servicing an accepted request that is not highpriority in order to immediately begin servicing a high priorityrequest. For example, at time point 2, the server 102 could suspendservicing the request from the first STB 302 and immediately beingservicing the high priority request from the third STB 306. Regardlessof the servicing order of the high priority requests, because the server102 continues to schedule requests having number of attempts less thanor equal to the threshold (i.e., first requests in this example) fromother STBs to be retried after a delay sufficient for the server 102 toexit the overloaded state, the server 102 maximizes its utilization,avoids building the congestion, and maintains at least a minimum levelof service to all client devices.

FIG. 6 is a flowchart describing operations performed by the server 102when servicing requests for information according to a predeterminedthreshold 117 representing the maximum number of acceptable attempts perrequest. The steps of the flowchart are not restricted to the exactorder shown, and, in other configurations, shown steps may be omitted orother intermediate steps added. In this configuration, the server 102performs the following operations:

Step 600: A request for information is received from a STB 104 by theserver 102.

Step 602: The server 102 determines whether or not the server 102 isoperating in an overloaded state. This may involve checking if thecommunication module 112 has exceeded or is approaching a maximum numberof concurrent TCP connections on the network, a maximum CPU/processor114 usage, a maximum network capacity, a maximum disk throughput, amaximum number of concurrent database connections, or other maximumprocessing ability of the server 102. If the server 102 is notdetermined to be operating in an overloaded state, control proceeds tostep 612; otherwise, if the server 102 is overloaded, control proceedsto step 604.

Step 604: The request number module 118 determines a number of attemptsfor the request received at step 600. For example, in onceconfiguration, each STB 104 will count the attempts using counter 108and include the current attempt count in a header of the connectionattempt. In another configuration, the server 102 stores counters 128 inits memory device 124 for counting the number of attempts for requestsby each STB 104.

Step 606: The number of attempts determined for the request is comparedwith a threshold 117 representing the maximum number of connectionattempts that could go denied by the server 102 due to being overloaded.In one configuration, the threshold 117 may be a fixed predeterminedvalue. If the number of attempts is not greater than the threshold 117,control proceeds to step 608; otherwise, control proceeds to step 612.

Step 608: A retry-after delay is calculated according to a predictedtime for the server 102 to exit the overloaded state. The retry-afterdelay takes into account pending requests including both requests beingcurrently serviced by the server 102 and requests that are alreadyscheduled to be serviced in the future. Calculating the retry-afterdelay may involve calculating how much time the server 102 will requirebased on a historical average rate of completing requests, how manyrequests are currently pending or being processed, how many requestshave been recently denied, or any combination of these. Another methodis to assign delays in a manner that will result in the retries byvarious STBs 104 being scheduled into individual timeslots in thefuture. For example, if three STBs make requests at substantially thesame time and the server 102 is too busy to process any of the requests,the server 102 could give the first STB a delay of 1 second, the seconda delay of 2 seconds, and the third a delay of 3 seconds. Predictivescheduling can also be utilized to estimate future loads and scheduleSTBs accordingly. For example, in a hotel that has 2000 STBs installedand the server 102 detects that a large number are requesting data atthe same time, the server 102 may schedule longer delays for STBs it isunable to service than if the hotel only had 500 STBs installed. Thereason is the server 102 can predict its future load level according tothe number of incoming requests, the type of requests, the total numberof STBs, etc. It is also possible to simply assign a random or fixeddelay according to how many total STB are currently installed in thehotel. For instance, if there are 100 STBs and each request typicallytakes 100 ms to process then the average delay could be fixed at 10seconds or chosen as a random delay up to 10 seconds.

Step 610: The connection with the requesting STB 104 is refused by thecommunication module 112 sending a service unavailable messagespecifying the retry-after delay to the STB 104.

Step 612: The connection with the requesting STB 104 is accepted and theserver 102 begins servicing the request according to a job priorityorder. The job priority order may be first come first serve, or may bebased upon how many times each request has been attempted, whererequests having a higher number of attempts are serviced before requestshaving a lower number of attempts.

FIG. 7 is a flowchart describing operations performed by the server 102when servicing requests for information according to a dynamic threshold117 representing the maximum number of acceptable attempts per request.Again, the steps of the flowchart are not restricted to the exact ordershown, and, in other configurations, shown steps may be omitted or otherintermediate steps added. In this configuration, the server 102 performssimilar steps to that of FIG. 6; however, in FIG. 7, the threshold 117is dynamically set in steps 720 to 724. In particular, in FIG. 7, theserver 102 additionally performs the following steps:

Step 720: The server 102 determines whether or not the server 102 isoperating in an overloaded state. This determination may be done in away similar to that described above for step 602. If the server 102 isdetermined to be operating in the overloaded state, control proceeds tostep 722; otherwise, if the server 102 is not operating in theoverloaded state, control proceeds to step 724.

Step 722: Because the server 102 is overloaded, the threshold 117 isdynamically set by the threshold setting module 120 to a value greaterthan zero. The exact value may be predetermined or automaticallydetermined according to formula I described above, for example.

Step 724: Because the server 102 is not overloaded, the threshold 117 isdynamically set by the threshold setting module 120 to a value of zero.This means that the first request attempt from a STB 104 will alreadyhave a number of attempts being greater than the threshold 117.Therefore, the server 102 will immediately accept all requestconnections and begin servicing the requests when it is not overloaded.

By giving higher priority service to STBs 104 that have been keptwaiting past a certain number of retries, the server 102 maintains abetter overall quality level. This is particularly useful in a hotelmedia system 100 when thousands of STBs 104 attempt to simultaneouslyload dynamic UI code and application software 110 upon boot-up or reset.According to one aspect of the invention, the server 102 indicates retrydelays to STBs 104 in hotel rooms to avoid network congestion and avoidincreasing server 102 overload, but will maintain a minimum level ofoverall service by giving priority to STBs 104 that have already mademany request attempts (i.e, more than a threshold 117).

Although the invention has been described in connection with a preferredembodiment, it should be understood that various modifications,additions and alterations may be made to the invention by one skilled inthe art without departing from the spirit and scope of the invention.For example, although the description of the invention has involved ahotel media system server 102 servicing requests from set-top boxes 104,the present invention is equally applicable to any hospitality relatedlocation or service wishing to provide any client devices withinformation from a server. Examples of hospitality locations include butare not limited to hotels, motels, resorts, hospitals,apartment/townhouse complexes, restaurants, retirement centres, cruiseships, busses, airlines, shopping centres, passenger trains, etc.Examples of client devices include set-top boxes, mobile phones, laptopcomputers, notebook computers, desktop computers, tablet computers,in-room control devices such lighting and heating devices, personaldigital assistants (PDAs), digital still and video cameras, and anyother device that may be utilized to request information from a server.Similarly, the present invention is also useful outside the hospitalityindustry for use to avoid overloading a server by a plurality ofclients.

In summary, a method of servicing requests for information at a serverincludes receiving a request for information from a client device, anddetermining a number of attempts of the request made by the clientdevice. Both the client device and the server may count the number ofattempts for each request. When the number of attempts is greater than athreshold, the method includes servicing the request and sending theinformation to the client device. When the number of attempts is notgreater than the threshold, the method includes sending a serviceunavailable message to the client device. A retry-after delay may becalculated by the server according to a predicted time for the server toexit the overloaded state and included in the service unavailablemessage. The predicted time may take into account both requestscurrently being serviced and requests that have been scheduled to beserviced in the future.

The various separate configurations, elements, features, and modules ofthe invention described above may be integrated or combined into singleunits. For example, a high definition television or other media platformcan be used instead of (or in addition to) a stand-alone set-top box. Inanother example, the various modules 112, 116, 118, 120, 122 and othercomponents 130, 114, 124 of the media system server 102 shown in FIG. 1may be located in different physical units. Similarly, functions ofsingle units may be separated into multiple units. Unless otherwisespecified, features and modules described may be implemented in hardwareor software according to different design requirements. Additionally,all combinations and permutations of the above described features andconfigurations are within the scope of the invention.

1. A method of servicing requests for information at a server, themethod performed by the server comprising: receiving a request forinformation from a client device; determining a number of attempts ofthe request made by the client device; servicing the request and sendingthe information to the client device when the number of attempts isgreater than a threshold; and sending a service unavailable message tothe client device when the number of attempts is not greater than thethreshold.
 2. The method of claim 1, further comprising dynamicallysetting the threshold according to an operating state of the server. 3.The method of claim 2, further comprising automatically setting thethreshold equal to a value being greater than zero when the server isoperating in an overloaded state.
 4. The method of claim 1, beingperformed when the server is operating in an overloaded state.
 5. Themethod of claim 4, further comprising: predicting an amount of timerequired to exit the overloaded state; and sending the serviceunavailable message to the client device including the predicted amountof time as a retry-after delay when the number of attempts is notgreater than the threshold; wherein, upon receiving the serviceunavailable message from the server, the client device is for waitingthe retry-after delay before attempting a subsequent retry of therequest.
 6. The method of claim 5, further comprising: predicting theamount of time required to exit the overloaded state by calculating anestimated time required to service pending requests; wherein the pendingrequests include current requests being serviced and prior requests fromother client devices that have not yet been serviced and for which theserver sent one or more service unavailable messages.
 7. The method ofclaim 1 further comprising accepting a connection with the client deviceand holding the connection in an open state until the information hasbeen sent to the client device when the number of attempts is greaterthan the threshold.
 8. The method of claim 7, further comprisingrefusing the connection with the client when the number of attempts isnot greater than the threshold.
 9. The method of claim 1, furthercomprising: determining the number of attempts of the request byexamining the request to read a current number of attempts as includedwithin the request by the client device; wherein the client devicecounts each attempt of the request and includes the current number ofattempts in the request.
 10. The method of claim 1, further comprisingdetermining the number of attempts of the request by counting at theserver a number of request attempts made by the client device.
 11. Aserver for servicing requests for information, the server comprising: acommunication module coupled to a network for receiving a request forinformation from a client device via the network; a request numbermodule for determining a number of attempts of the request made by theclient device; and a servicing module for servicing the request andsending the information to the client device when the number of attemptsis greater than a threshold, and for sending a service unavailablemessage to the client device when the number of attempts is not greaterthan the threshold.
 12. The server of claim 11, further comprising athreshold setting module for dynamically setting the threshold accordingto an operating state of the server.
 13. The server of claim 12, whereinthe threshold setting module is for setting the threshold equal to avalue being greater than zero when the server is operating in anoverloaded state.
 14. The server of claim 11, wherein the servicingmodule is for sending the service unavailable message to the clientdevice when the number of attempts is not greater than the threshold andthe server is operating in an overloaded state.
 15. The server of claim14, further comprising: a time prediction module for predicting anamount of time required to exit the overloaded state; wherein theservicing module is further for sending the service unavailable messageincluding the predicted amount of time as a retry-after delay when thenumber of attempts is not greater than the threshold; and upon receivingthe service unavailable message from the server, the client device isfor waiting the retry-after delay specified in the service unavailablemessage before attempting a subsequent retry of the request.
 16. Theserver of claim 15, further comprising: a pending request log forstoring a record of pending requests including current requests beingserviced and prior requests from other client devices that have not yetbeen serviced and for which the servicing module sent one or moreservice unavailable messages; wherein the time prediction unit isfurther for predicting the amount of time required to exit theoverloaded state by calculating an estimated time required to servicethe pending requests.
 17. The server of claim 11, wherein, when thenumber of attempts is greater than the threshold, the communicationmodule is further for accepting a connection with the client device andholding the connection in an open state until the information has beensent to the client device.
 18. The server of claim 17, wherein, when thenumber of attempts is not greater than the threshold, the communicationmodule is further for refusing the connection with the client.
 19. Theserver of claim 11, wherein the request number module is for determiningthe number of attempts of the request by examining the request to read acurrent number of attempts as included within the request by the clientdevice; the client device counting each attempt of the request andincluding the current number of attempts in the request.
 20. The serverof claim 11, further comprising: a memory device having stored therein acounter; wherein the request number module is for determining the numberof attempts of the request by utilizing the counter to count a number ofrequest attempts made by the client device.